AVRAJIT GHOSH
Embedded Files
I am a postdoc fellow at Simons Institute for the Theory of Computing and BAIR (UC Berkeley EECS). I am affiliated with the Machine Learning Research Pod where I am fortunate to be advised by Peter Bartlett and Bin Yu.
My PhD thesis is titled Implicit Regularization of Hyperparameters in Deep Learning: Beyond Convexity and Small Steps. Through four chapters, the thesis explores the practical role of 1) learning-rate, 2) momentum, 3) weight perturbation and 4) architecture in implicitly regularizing the hypothesis space in deep learning. I graduated from Computational Mathematics Science and Engineering department at Michigan State University where I was fortunate to be advised by Rongrong Wang and Saiprasad Ravishankar.
My research aim is to bridge the gap between practical deep learning and theory by investigating the training dynamics across various deep learning tasks and to develop a theoritical framework to understand the dynamics.
Research Publications: (* denotes equal contribution)
Training dynamics in deep learning: Role of Hyperparameters
1) Variational Learning Finds Flatter Solutions at the Edge of Stability [Learning-rate + weight perturbation]
1) Variational Learning Finds Flatter Solutions at the Edge of Stability [Learning-rate + weight perturbation]
Avrajit Ghosh, Bai Cong, Rio Yokota, Saiprasad Ravishankar, Rongrong Wang, Molei Tao, Mohammad Emtiyaz Khan, Thomas Möllenhoff
(arXiv preprint, under submission)
Avrajit Ghosh*, Soo Min Kwon*, Rongrong Wang, Saiprasad Ravishankar, Qing Qu
(ICLR-2025)
3) Implicit regularization in Heavy-ball momentum accelerated stochastic gradient descent. [Learning-rate + momentum]
Avrajit Ghosh*, He Lyu*, Xitong Zhang, Rongrong Wang
(ICLR-2023, Spotlight, Top 6%)
Generalization theory in deep learning
1) PAC-Bayes Generalization bounds for Score Based Diffusion Models
Avrajit Ghosh, Rongrong Wang
(under submission)
2) Improving Generalization of Complex Models with Unbounded Loss Using PAC-Bayes Bounds.
Xitong Zhang, Avrajit Ghosh, Guangliang Wang, Rongrong Wang
(TMLR-2024)
Inverse problems/ Compressed sensing
1) Optimal Eye Surgeon: Finding Image Priors through Sparse Generators at Initialization. [Architecture]
Avrajit Ghosh, Xitong Zhang, Kenneth Sun, Qing Qu, Saiprasad Ravishankar, Rongrong Wang
(ICML-2024)
2) Learning Sparsity Promoting Regularizers using Bilevel Optimization.
Avrajit Ghosh, Michael Mccann, Madeline Mitchell, Saiprasad Ravishankar
(SIAM Journal on Imaging Sciences - 2024)
3) Understanding Untrained Deep Models for Inverse Problems: Algorithms and Theory.
Avrajit Ghosh*, Ismail Alkhouri*, Evan Bell*, Shijun Liang, Rongrong Wang, Saiprasad Ravishankar
(IEEE SPM Special Issue on the Mathematics of Deep Learning-2025)
4) Bilevel Learning of L1 Regularizers with Closed-Form Gradients.
Avrajit Ghosh, Michael Mccann, Saiprasad Ravishankar
(ICASSP- 2022)
Page updated
Google Sites
Report abuse